Prospective Evaluation of a Rapid Clinical Metagenomics Test for Bacterial Pneumonia

Background The diagnosis of bacterial pathogens in lower respiratory tract infections (LRI) using conventional culture methods remains challenging and time-consuming. Objectives To evaluate the clinical performance of a rapid nanopore-sequencing based metagenomics test for diagnosis of bacterial pathogens in common LRIs through a large-scale prospective study. Methods We enrolled 292 hospitalized patients suspected to have LRIs between November 2018 and June 2019 in a single-center, prospective cohort study. Rapid clinical metagenomics test was performed on-site, and the results were compared with those of routine microbiology tests. Results 171 bronchoalveolar lavage fluid (BAL) and 121 sputum samples were collected from patients with six kinds of LRIs. The turnaround time (from sample registration to result) for the rapid metagenomics test was 6.4 ± 1.4 hours, compared to 94.8 ± 34.9 hours for routine culture. Compared with culture and real-time PCR validation tests, rapid metagenomics achieved 96.6% sensitivity and 88.0% specificity and identified pathogens in 63 out of 161 (39.1%) culture-negative samples. Correlation between enriched anaerobes and lung abscess was observed by Gene Set Enrichment Analysis. Moreover, 38 anaerobic species failed in culture was identified by metagenomics sequencing. The hypothetical impact of metagenomics test proposed antibiotic de-escalation in 34 patients compared to 1 using routine culture. Conclusions Rapid clinical metagenomics test improved pathogen detection yield in the diagnosis of LRI. Empirical antimicrobial therapy could be de-escalated if rapid metagenomics test results were hypothetically applied to clinical management.


INTRODUCTION
Lower respiratory tract infection (LRI) is one of the top four causes of mortality worldwide (Naghavi et al., 2017). However, the identification of causative agents of LRI remains challenging due to the limitations of the current methodology. Conventional methods for diagnosing LRI, mainly using culture and serological tests, are insensitive and time-consuming (Holter et al., 2015). As a result, pathogens were only identified in 38% of adults who presented the radiographic evidence of pneumonia (Jain et al., 2015). The abuse of broad-spectrum antibiotics has made it even harder to identify pathogens, as patients may have already received antibiotics before the tests. A delayed diagnosis leads to inappropriate empiric, broad-spectrum antibiotic therapy, which causes poor therapy outcomes, longer hospital stays, and higher costs (Vaughn et al., 2019;Webb et al., 2019).
Unlike culture methods, molecular techniques identify pathogens by genetic molecules instead of microbe clones (Varadi et al., 2017). Although targeted techniques such as PCR are fast, they only allow the identification of carefully chosen pathogens (Hassibi et al., 2018;Poritz and Lingenfelter, 2018). Clinical metagenomics uses next generation sequencing of total nucleic acid from clinical samples to detected all the microbes simultaneously, allowing for unbiased pathogen identification that is less affected by clinical prejudgement (Goldberg et al., 2015;Forbes et al., 2017;Blauwkamp et al., 2019;Wilson et al., 2019). Superior in terms of rapid library preparation and real-time data acquisition and analysis, the nanopore sequencing platform (Nanopore, Oxford, UK) has proven its ability to rapid LRI pathogen detection in recently studies (Charalampous et al., 2019). However, most previous studies were limited to individual patient or a series of small, retrospective cases (Pendleton et al., 2017;Moon et al., 2018;Yang et al., 2019). A question remains unclear: what is the performance and potential clinical value of applying rapid metagenomics in the diagnosis of common respiratory infections?
This was a prospective, single-center, on-site study involving hospitalized patients with community-acquired pneumonia (CAP), community-acquired pneumonia in immunocompromised host (CAP-ICH), hospital-acquired pneumonia (HAP), acute exacerbation of bronchiectasis (AEBX) (Woodhead et al., 2011), acute exacerbation of chronic obstructive pulmonary disease (AECOPD), and lung abscess, which were diagnosed based on guideline from American Thoracic Society, Chinese Thoracic Society, Infectious Diseases Society of America (Supplementary File E1). The study aimed to evaluate the clinical performance of a commercial rapid metagenomics test (Simcere Diagnostics, Nanjing, China), including turnaround time, pathogen identification rate, sensitivity, and specificity of on-site rapid metagenomic testing on bronchoalveolar lavage (BAL) or sputum samples collected from patients with LRIs.

Ethics Statement
The study was carried out in China-Japan Friendship Hospital, Beijing, China. Ethical approval was obtained from the China-Japan Friendship Hospital Ethics Committee(2018-145-k102). All subjects provided written consents.

Study Design
Between November 2018 and June 2019, a cohort of 292 consecutively hospitalized patients suspected to have LRIs, including CAP, CAP-ICH, HAP, AEBX, AECOPD, and lung abscess, was enrolled after meeting the following inclusion criteria: Age ≥14 years, recent-onset/worsening cough, dyspnea, tachypnea, recent purulence or change in sputum characteristics, increased secretions or suctioning requirements, radiographic findings of new, progressive or persistent infiltrate (Table 1). BAL samples were collected from patients if bronchoscopy is necessary. Sputum was qualified by microscopy examination of gram staining slide: a sputum sample was qualified if they had <10 squamous cells and >25 leukocytes per low-power (×10) field. The on-site study involved collection of either BAL or qualified sputum sample, and parallel testing of these samples using routine microbiological techniques and rapid metagenomics.

Reference Standard for Microbiological Diagnosis in This Study
Reference standard for microbiological diagnosis was defined as any positive result on routine microbiological culture, urinary antigen tests, or qPCR and Sanger sequencing tests. Respiratory tract samples from all 292 patients with LRIs underwent routine culture during their hospital stay. Urinary antigen tests were used for detection of Streptococcus pneumoniae. When rapid metagenomics results were discordant with culture or urinary antigen results, these pathogens were further verified by qPCR and Sanger sequencing (Supplementary Table E1). The primers and probes for qPCR and Sanger sequencing were shown in the Supplementary Method.

Clinical Relevance and Appropriateness of Therapy
Patients' medical records were assessed to determine whether the pathogens reported by rapid metagenomics were the potential cause of the clinical presentation. The appropriateness of the treatment regimen was assessed considering the treatment outcome and antibiotic regimen. The prescribed antimicrobial for each patient was compared with the hypothesized antimicrobial(s) which would be appropriate for pathogendirected therapy based on the pathogen identified by metagenomic sequencing. The clinical features, radiologic and laboratory findings, antimicrobial use, and clinical improvement of each patient were independently reviewed by two clinicians (Dr. YL and XC). All patients were followed until discharge or death.
contaminations eliminated was defined as a Meta-ID (Supplementary Figure E2 and Method).

Statistical Evaluation of Pathogen Identification Capacity
Meta-IDs on the common pathogen list used by routine microbiological tests were compared to the reference standard (Supplementary Figures E1, E3), to obtain statistics on diagnostic performance. Sensitivity, specificity, positive predictive value, negative predictive value, confidence intervals, positive concordance, negative concordance and pathogen identification rate were calculated using R software (v3.6.0). Gene set enrichment analysis (GSEA v4.0.3) (Subramanian et al., 2005) was performed on Meta-IDs against 6 types of diseases. See Supplementary Methods for more details.

Characteristics of Rapid Clinical Metagenomics Test
Each collected sample was divided into two aliquots for simultaneous routine microbiological testing and on-site rapid metagenomic sequencing. BAL samples were collected from 171 subjects (59%), and qualified sputum samples were collected from the remaining subjects ( Figure 1A). The library construction method was optimized based on the fastest library preparation kit (ONT, Oxford, UK), making the total time required for library preparation, including sample DNA purification and loading, was less than 1 hour. In addition, as sequencing data are generated in real time during bioinformatics analysis, the reporting time was ≤ 4 hours when sufficient data were accumulated for pathogen identification. As the time required for microbe read number > 1,000 is typically less than 4 hours for some samples, the sequencing process was completed before the designed end-time and the median sequencing duration was 2.6 hours ( Figure 1B). The overall turnaround time was 6.4 ± 1.4 hours. In contrast, the overall turnaround time for routine microbiological testing was 94.8 ± 34.9 hours (Supplementary Figure E4). The median read length of each sample is shown in Figure 1C. All reads used in this study were longer than 500 bp and most reads were in the range of 1,000-3,000 bp, with a median length of 1,152 bp. Furthermore, a previous study and our simulation analysis have demonstrated that the accuracy for identification of species is enhanced with longer read-lengths, even with relative poor single-base accuracy (Supplementary Figure E5).
We tested several threshold to define a Meta-ID and the result showed the threshold we chose is robust (Supplementary Figure E2). The abundance of Meta-IDs was highly related to its verification status ( Figure 1D). In total, the abundance of all Meta-IDs ranged from 1% to 100%, with a median abundance of 20.6%. Meta-IDs of pathogens in culture-positive samples typically dominated or were in high abundance, with a median abundance of 69.1%; Meta-IDs missed in culture methods but verified by validation test exhibited medium abundance, with a median value of 31.0%, while Meta-IDs of unverified pathogen showed a median abundance of 25.0%; Meta-IDs PCT≥0.25 ng/mL, n (%) 175 (59.9) White blood cell count,median (IQR), × 10 9 /L 8.4 (6.0, 12.5) Neutrophil count,median (IQR), × 10 9 /L 6.4 (4.1, 10.4) Lymphocyte count,median (IQR), × 10 9 /L 1.1 (0.6, 1.6) Platelet count,median (IQR), × 10 9 /L 206.0 (150.0, 284.0) Hemoglobin,median ( Among the 22 patients with Lung Abscess, 2 were immunocompromised hosts. c Immunosuppression: including 27 longterm steroid use, 24 receiving cancer chemotherapy, 5 solid organ transplantation, 2 primary immune deficiency diseases,2 receiving anti-rheumatic drugs or other immunosuppressive drugs. verified using qPCR or sanger sequencing but culture-negative were typically in relatively low abundance of 20.0%, indicating the failure of culture methods possibly due to low abundance; Meta-IDs of experimentally-unverified pathogens in culture-negative samples were in low abundance, with a median abundance of 14.1% ( Figure 1D).
In total, 80 microbe species were identified by rapid metagenomics (Supplementary File E3), among which about half of them are Gram-stain negative and another half is Gramstain positive, with the exception of Chlamydia psittaci, Mycoplasma pneumoniae, and Mycoplasma hominis, which are not visualized by Gram-staining ( Figure 1E). Meanwhile, anaerobes accounted for 49% of the total species identified by rapid metagenomics while none of them were reported by routine microbiological test ( Figure 1F). Rapid metagenomics failed in 2 samples which reported Acinetobacter nosocomialis and Leclercia adecarboxylata by culture methods (Supplementary Table E2). Both species were found in the sequencing raw data but did not pass the Meta-ID criteria. To sum up, pathogens in 45% (n=131) were identified by culture methods, 58% (n=169) were reported by rapid metagenomics, 41% (n=119) were reported by both methods, and 17% (n=50) were verified by qPCR tests ( Figure 1G). Pathogens in 2% (n=5) of the samples were only reported by culture but failed to be verified by qPCR tests. Pathogens in another 2% (n=5) of the samples were reported and verified by qPCR tests but not reported by rapid metagenomics. Both methods failed in 34% (n=98) of the total patients. Among ICU patients, pathogens were identified in 67% of the them using culture method plus rapid metagenomics. Furthermore, the culture method plus rapid metagenomics identified pathogens in 82% of HAP patients (Supplementary Figure E6).

Performance of the Rapid Metagenomics Compared With Traditional Methods
In culture-positive samples, rapid metagenomics achieved a concordance ratio of 92.4% (121/131) in comparison with culture results ( Table 2). With regards to the 10 samples with discordant results, 5 samples failed to be verified by qPCR tests; 3 samples showed low DNA concentrations (0.13, 0.87, and 1.3 ng/mL, respectively), and 2 samples failed to pass the predefined thresholds in metagenomic method. In culture-negative samples, rapid metagenomics achieved a concordance ratio of 60.9% (98/161), in comparison with culture results ( Table 2). For samples with discordant results, rapid metagenomics identified pathogens in all samples and 79.4% (50/63) of them had been experimentally verified, which improved the concordance ratio to 91.9% (148/161) compared with culture and validation results together. The remaining 13 samples failed to be verified by validation experiment. Overall, rapid metagenomic achived a sensitivity of 96.6%, specificity of 88.0%, overall positive predictive value (PPV) of 92.3%, and negative predictive value (NPV) of 94.5% (Table 3). Among the five LRI diseases, CAP achieved the highest performance, with sensitivity of 97.6%, specificity of 90.2%, PPV of 91.1%, and NPV of 97.4%. Diagnostic performance was similar between patients in ICU and those in the general ward, with sensitivity of 96.0% to 97.4%, specificity of 86.3% to 89.4%, PPV of 93.1% to 91.4%, NPV of 91.7% to 96.7%, respectively.

More Fastidious Pathogens Identified by Rapid Metagenomics
Identification fastidious pathogens are challeging for traditional methods, especially after exposure to antibiotics. Here we tested rapid metagenomics in three most common fastidious pathogens (Carroll, 2002;Kollef, 2006;Moran et al., 2013) i.e., Streptococcus pneumoniae, Haemophilus influenzae, and Moraxella catarrhalis. Rapid metagenomics identified fastidious pathogens in 37 cases, including S. pneumoniae (n=16), H. influenzae (n=12), and M. catarrhalis (n=9); while traditional methods identified in 13 cases, including 6 cases of S. pneumoniae, 3 cases of H. influenzae, and 4 cases of M. catarrhalis. Among the 11 cases with discordant results on S. pneumoniae, 7 were validated by qPCR, 3 failed, and 1 was not reported by rapid metagenomics as its abundance was below the cutoff value (only 1 reads in 4 hour data). The abundance of S. pneumoniae was relatively higher in samples with verified results than those without ( Figure 2A). Moreover, samples with unverified S. pneumoniae tended to be dominated by high abundance of other commensal Streptococcus species. In total, S. pneumoniae was identified and verified in 5 patients with CAP, 2 patients with AECOPD, 2 patients with AEBX, 2 patients with HAP, 2 patients with CAP-ICH, and 1 patients with lung abscess ( Table 4). One patient with CAP-ICH had their therapy deescalated because clinicians were made aware of a positive S. pneumoniae urinary antigen test.
All 12 patients positive for H. influenzae by metagenomics were also positive by the reference methods, i.e., 3 by culture method and 9 by qPCR, with a median abundance of 58% of the microbial reads. All 9 patients positive for M. catarrhalis by metagenomics were also positive by reference standard, plus 4 were verified by culture and 5 by PCR, with a median abundance of over 87% of the microbial reads (Figure 2A). H. influenzae was identified and verified in 12 patients including 2 with AECOPD, 5 with AEBX, 2 with HAP, and 1 with CAP-ICH, while M. catarrhalis was identified and verified in 9 patients, including 1 with CAP, 6 with AECOPD, 1 with AEBX, and 1 with CAP-ICH. Broad-spectrum antibiotics were empirically used in most patients, and no de-escalation was achieved as clinicians were not aware of the causative pathogens before prescription ( Table 4).

Higher Potential to Detect Anaerobic Species
Significantly more anaerobic species were identified by rapid metagenomics than by culture techniques (49% and 0%, respectively). Among the six types of LRI involved in this study, a large portion of anaerobic species (43%) and a small portion of aerobic species (27%) were identified in patients with lung abscess ( Figure 2B). Subsequently, Gene Set Enrichment Analysis (GSEA) showed that anaerobic species were significantly enriched while aerobic species were significantly depleted in patients with lung abscess, with FDR < 0.25 ( Figures 2C, D). As many of the anaerobic species are commensal bacteria in the upper respiratory tract, we performed additional enrichment analysis and the results showed similar association (Supplementary Figure E7). Finally, among the 50 patients with negative culture results, metagenomic results could be interpreted in 33 patients, as the probable or possible cause of their clinical presentation (Supplementary File E5) by adjudication definitions (Blauwkamp et al., 2019).

Detection of Respiratory Viruses in Patients
Besides bacterial pathogens identified in these patients, viral infections were also detected in our patients. By reviewing clinical laboratory findings, among patients with both methods failed (n=98), 42% of patients got at least one virus detected in the 74 patients who were subjected to respiratory virus test. Among patients with positive culture results (n=131), 86 were tested for respiratory virus and 50% of them got at least one virus detected. Of the patients with positive results in nanopore test (n=169), 51% were positive for respiratory viruses among the 113 patients tested (Supplementary File E4). The most frequently detected viruses were Epstein-Barr virus (EBV) (n=47), human cytomegalovirus (HCMV) (n=33), influenza A (n=28), respiratory syncytial virus (RSV) (n=10), parainfluenza virus (n=2), adenovirus (ADV) (n=2), influenza B (n=1).

DISCUSSION
This study enrolled nearly three hundred patients to evaluate the performance of rapid metagenomics for the diagnosis of bacterial pathogens in LRI in the real-life scenarios, including patients from both ICU and general wards. Our study showed rapid metagenomics could improve the diagnositic yield for LRI, especially on pathogens that are difficult to culture, such as fastidious bacteria or anaerobes. For example, among the 34 patients with the three most common fastidious pathogens identified by rapid metagenomics, only 13 were reported by conventional methods. In fact, if the results of the metagenomics test had been used to guide therapy, 33 patients could have had their empiric therapy correctly de-escalated. In total, rapid metagenomics achieved an overall sensitivity of 96.6%, specificity of 88.0%, PPV of 92.3%, and NPV of 94.5%; the sensitivity and specificity were similar between patients from ICU and those from general wards. These performance characteristics make rapid metagenomics a very attractive solution for the rapid diagnosis of LRI. Rapid metagenomics identified 38 anaerobic bacterial species in 49% samples, while none of the anaerobes were identified by culture. Although previous studies suggested that anaerobic bacteria are responsible for lung abscess (Hammond et al., 1995;Bartlett et al., 2000;Takayanagi et al., 2010), few anaerobes are cultured in clinical practice, and the treatment for lung abscess is depended on empirical broad-spectrum antibiotics without knowledge of the causative pathogens. Moving forward, if rapid clinical metagenomics was applied broadly on LRIs diagnosis, clinicians will have a growing amount of evidences identifying the bacteria associated with aspiration pneumonia and lung abscess, which is likely to increase the recovery rate of antibiotic therapy in these infections.
The turnaround time, from sample registration to pathogen identification, is much shorter for rapid metagenomics (6.4 ± 1.4 hours) than conventional culture-based diagnostics (94.8 ±34.9 hours). Considering the high mortality of ICU patients with severe LRI, a rapid diagnosis of causative pathogen is crucial for timely and appropriate antimicrobial therapy (Jain et al., 2015). With regards to the choice of antibiotics given to ICU patients with severe LRIs, intensivists believe that "broader is safer" (Waterer, 2019). Although antibiotics may be adjusted after the initial 72 hours of therapy (American Thoracic and Infectious Diseases Society of, 2005), intensivists are always reluctant to deescalate without knowledge of causative pathogens (Morel et al., 2010;Joung et al., 2011;Heenen et al., 2012;Garnacho-Montero et al., 2014;Garnacho-Montero et al., 2015). The short turnaround time of rapid metagenomics will allow intensivists to be more confident in suspending broad-spectrum antibiotics earlier before causing additional adverse effects in patients. The current rapid metagenomic workflow required batch sequencing of several samples on one flowcell. To achieve the minimum    turnaround time, metagenomic sequencing could be applied on low-throughput flowcell, such as Flongle, when a small number of sample need to be tested.
Though metagenomic sequencing achieved higher diagnositic yield, we still got 34% of samples with no pathogens identified in both metagenomic sequencing and traditional methods; we peculated that these patients might be of viral infection or noninfection respiratory diseases, as 42% of patients got respiratory virus detected. Furthermore, 50% of patients had both positive culture result and at least one virus detected. Interactions between viruses and bacteria in the pathogenesis of respiratory infections have been extensively reported in previous reports (Hament et al., 2004;Verkaik et al., 2011). For example, respiratory viruses could promote bacterial adhesion to respiratory epithelial cells, a process that may increase bacterial colonization and contribute to disease (Avadhanula et al., 2006). This findings suggested that viral infection should be considered in the design of future clinical metagenomic pipeline.
Limitations of this study should be noted. Firstly, more than 85% of the enrolled subjects had already received antibiotics before the collection of BAL or sputum samples, which may lead to bias in negative report of both culture methods and rapid metagenomic sequencing. Secondly, detecting a potential pathogen dosesn't mean that we catched the causing agent of the diseases, as potential pathogens can be detected frequently in respiratory samples from people without respiratory symptoms. Thus, the full clincal pictures of the patients should be reviewed to determine the causive agents of LRTIs, including serum biomarker for infection, radiographic imaging, and clinical manifestation of patients. In our study, we involved two clincians to evaluate the potential role each species may play in the process of the disease, based on patient's clinical data and guideline issued by ATS, CTS, IDSA and ESCMID (Takayanagi et al., 2010;Liu et al., 2012;Musher and Thorner, 2014;Di Pasquale et al., 2019). Moreover, the cost of test should be considered on the real clinical settings, as this rapid metagenomic test costed about 500 $ per sample, which is much higher than that of traditional methods. Currently, metagenomic test is unlikely to replace conventional diagnostics in the near future but can be a complementary diagnositcs approach in certain clinical situations (Chiu and Miller, 2019), such as novel infectious diseases outbreak, critical patients with unexplained pathogens, and immunocompromised patients who are easily infected by uncommon pathogens which are not covered in conventional methods. (Xie et al., 2019;Yang et al., 2019;Wang et al., 2020;Azar et al., 2021). For these patients, timely detection of causing pathogens by the rapid metagenomic test could prevent delayed and inadequate therapy, prolonged stays, increased costs, and high mortality and morbidity (Graf et al., 2016).
In conclusion, we reported a rapid clinical metagenomics test for the untargeted detection of bacterial LRIs. It has been demonstrated to be faster and more sensitive than traditional diagnoisis methods, especially for fastidious and anaerobic bacteria, providing an understanding of the microbial community present in the respiratory sample and the relative abundance of the pathogen in that community. There is an urgent need for further carefully designed research to provide scientific evidences for the patient management and economic benefits that offered by this new technology in the clinical setting.

DATA AVAILABILITY STATEMENT
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary Material.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by China-Japan Friendship Hospital Ethics Committee (2018-145-k102). The patients/participants provided their written informed consent to participate in this study.